Standardized tests of mental competency are widely used in a variety of applications. In addition to the well-known college entrance examinations, such tests are used to measure student progress through elementary and secondary school years, to verify competence for entry into the practice of a profession, such as medicine, law, accounting, real estate, etc., to verify minimum competence to posses a license or permit, such as a driver's license, and for many other uses. As used herein, “mental competency test” refers broadly to any of various types of tests which ask questions of a test subject and require communicative responses, whether such tests measure knowledge of facts, memory, analytical ability, aptitude, mental speed, linguistic capabilities, or other intellectual attributes, either alone or in combination. Such tests are often given in a multiple-choice format, because multiple-choice provides advantages of ease of scoring and comparison of results.
In most (although not necessarily all) such applications, it is expected that the test subject will not know the specific questions and answers in advance of taking the test. Having advance knowledge of either the questions or the answers would give the test subject a significant advantage with respect to other test subjects, and would undermine the integrity of the testing process itself. Therefore, testing authorities go to considerable lengths to keep test materials secret from potential test subjects. Unfortunately, security breaches do sometimes occur, and test subjects sometimes gain unauthorized access to test materials in advance of taking a test.
One common testing security technique is to assure that all test subjects take the test at the same time. I.e., a particular version of the test is prepared for use on a particular date and time, and simultaneously distributed to all test subjects at the appointed time. Prior to distribution, the test questions and answers are closely guarded secrets, enmeshed in considerable physical security. In general, this technique can provide a high level of security, but the requirement that all subjects take the test at the same time may be impractical in certain circumstances.
In many environments, it is a practical necessity to offer the test on a continuing basis or frequently scheduled basis. In these environments, it is not generally practical to change the test version each time a test is offered. A test subject who takes the test in these environments may walk away from the test with knowledge that would be useful to another subject taking the test a few hours, a few days, or a few weeks later.
One common application for mental competency tests is the use of certification examinations by various private companies acting as testing authorities, which are intended to assure that the test subject is competent to perform some set of tasks. In particular, a variety of certification examinations are used in the computer industry to assure competency to maintain or administer computer systems, perform repairs, design custom installations, write custom application programming code, and so forth. Such certification tests are designed and maintained by the companies themselves, and due to the economic realities of hiring and training, are generally offered on a continuing basis and in a variety of dispersed geographic locations. Often, obtaining a job or promotion, or keeping an existing position, is dependent on successful completion of a certification examination. These examinations are therefore of considerable importance to the individual being tested.
An exposure exists where knowledge gained by one or more test subjects taking a test may be later imparted to other test subjects taking the same test, or a subsequent version of the same test. While this exposure exists for any form of mental competency test, it is particularly acute in the case of certification tests, which are offered on a continuing basis.
A legitimate test subject normally remembers some aspects of the test, such as the general subject areas, types of questions, level of difficulty, and so forth. Such a test subject will usually remember a handful of questions in incomplete detail, i.e., he may remember the gist of the question and what he thought to be the correct answer. It is difficult or impossible to prevent this knowledge from being taken away and potentially shared with other test subjects. But such knowledge is not likely to significantly skew the results of a well-designed test. The details which will be remembered are too few, and these will not always be remembered accurately.
But in some circumstances, unscrupulous individuals have obtained complete or nearly complete copies of a test in advance. This might be accomplished, e.g., with collusion of a test proctor, or by a test subject photographing the test using a miniature camera, and later providing the test to others. Other systematic methods of obtaining test questions and answers might also be used.
If a test subject is provided with a complete or nearly complete copy of test questions and correct answers in advance, that subject can simply memorize the correct answers, without having to master the material. The subject's score on the test will not reflect the true state of the subject's competence intended to be tested.
One known method for reducing this exposure is to use pools of questions, the pool being substantially larger than the number of questions on any one test, so that when giving a test, a subset of questions is selected from the pool. The use of computerized testing makes it possible to change the subset of questions each time a test is given. However, creating a sufficiently large question pool increases the cost of maintaining the test. The cost is particularly high in the computer industry, where new versions of systems, devices and applications are constantly being introduced.
A need exists for improved techniques for discouraging and/or detecting improper access to text questions and answers.